Image attribute discrimination apparatus, attribute discrimination support apparatus, image attribute discrimination method, attribute discrimination support apparatus controlling method, and control program

ABSTRACT

An attribute of image data can accurately be discriminated. An image attribute discrimination apparatus includes a heterogeneous region extracting unit that specifies a heterogeneous region from image data. The heterogeneous region includes a heterogeneous matter whose attribute is different from that of a content originally produced by the image data. An image attribute discrimination apparatus further includes a scene discrimination unit that discriminates the attribute of the image data based on a feature quantity extracted from a pixel group except each pixel in the heterogeneous region in each pixel of the image data.

BACKGROUND OF THE INVENTION

1. Technical Field

One or more embodiments of the present invention relate to imageattribute discrimination processing of discriminating an attribute ofdigital-format image data such as a still image and a moving image,particularly to an image attribute discrimination apparatus, anattribute discrimination support apparatus, an image attributediscrimination method, an attribute discrimination support apparatuscontrolling method, and a control program for improving accuracy of theimage attribute discrimination processing.

2. Related Art

Recently, there is conducted research and development of a technique ofanalyzing a feature of image data to automatically discriminate anattribute of the image data. Specifically, a feature quantity isextracted from a pixel value possessed by any piece of image data suchas the still image or moving image which is imaged with a digitalcamera, a digital video camera, or a camera-equipped mobile phone, thestill image captured by a scanner, and the moving image or capture imagewhich is recorded by a DVD recorder, and a scene (attribute) expressedby the image data is discriminated. For example, what kind of scene(such as person, landscape, night view, sunset, firework, room interior,snow, beach, flower, cooking, and business card and document) taken bythe real-time image data processed by the digital camera isdiscriminated, which allows a photograph to be taken while aphotographing mode of the digital camera is set to an optimum stateaccording to the scene.

For example, Japanese Unexamined Patent Publication Nos. 11-298736(published on Oct. 29, 1999), 2002-218480 (published on Aug. 2, 2002),2005-310123 (published on Nov. 4, 2005), and 2005-122720 (published onMay 12, 2005) disclose known image attribute discrimination processingtechniques. In the techniques disclosed in Japanese Unexamined PatentPublication Nos. 11-298736, 2002-218480, 2005-310123, and 2005-122720,the feature quantity is extracted from target digital image data toperform processing of checking the feature quantity against apreviously-prepared model feature quantity with respect to a specificscene, and a scene is discriminated based on a degree of coincidencewith the feature quantity of the specific scene.

More specifically, in an image processing apparatus disclosed inJapanese Unexamined Patent Publication No. 11-298736, a determinationwhether the image data is the sunset scene is made using a histogram ofhue data, and a determination whether the image data needs to becorrected is made based on the determination whether the image data isthe sunset scene. The image processing apparatus makes the histograms ofa value a product of the hue and chroma and a value of a product of thehue and lightness with respect to the pixels belonging to a range of redto yellow in the pixels constituting the target image data, and theimage processing apparatus determines that a variance of the histogramthat is larger than a specific reference is the image of the scene“sunset”.

Japanese Unexamined Patent Publication No. 2002-218480 discloses animage photographing apparatus that discriminates a plurality of scenessuch as “portrait”, “sunset”, and “night view” with respect to thetarget image data with information on the presence or absence of aperson and information on a color histogram as a common feature index.

Japanese Unexamined Patent Publication No. 2005-310123 discloses anapparatus that accurately selects various images of specific scenes withrespect to a feature portion corresponding to the specific scene inconsideration of tendency of disposition in the image and inconsideration of a position of photographing frame and a variation ofarea ratio by a photographing frame taking difference.

In an apparatus disclosed in Japanese Unexamined Patent Publication No.2005-122720, reference data in which a kind of the feature quantity andan identifying condition are defined is prepared in each of theplurality of scenes designated as the specific scene in order toidentify the scene, and scene discrimination is accurately performed byreferring to the identifying condition.

However, in the conventional configurations, unfortunately the attributecannot correctly be discriminated when a substance, a shadow, and ashape which are different from those of the original attribute of theimage data (hereinafter referred to as a heterogeneous matter) areincluded in the image data that becomes the attribute discriminationtarget. That is, the feature obtained from a pixel group (hereinafterreferred to as a heterogeneous region) taking the heterogeneous matteris different from the feature of the original attribute. Therefore, whenthe feature quantity of the whole image data is extracted while thefeature quantity of the heterogeneous region is mixed in the featurequantity of the image data, the checking against the model featurequantity is not successfully performed, which results in false scenediscrimination is performed to the image data or scene discrimination isperformed with low likelihood.

For example, the generation of the heterogeneous region in the imagedata is attributed to objects, (also includes telop in the case of themoving image) such as a character, an illustration, a graphic, a stamp,and a graffiti, which are added to the image data that becomes theattribute discrimination target using an image edit tool in an imageedit process. Additionally, sometimes an unintended phenomenon (such asa white spot phenomenon such as smear) emerges in the image dataphotographing process depending on a photographing environment or asubject state or an intended body such as a finger shadow is taken inthe photograph. Additionally, sometimes an original plate or a backsidecolor of an original is taken in a lack portion of the original in theprocess of scanning the origin such as the photograph (due to the brokenoriginal or folded original). The heterogeneous region is not limited tothe above-described examples. The above problems are commonly generatedirrespective of the condition, environment, and situation relating tothe image data, when the attribute discrimination is performed to anypiece of image data including the heterogeneous matter whose attributeis different from that of the original scene.

SUMMARY

One or more embodiments of the present invention realize an imageattribute discrimination apparatus, an attribute discrimination supportapparatus, an image attribute discrimination method, an attributediscrimination support apparatus controlling method, and a controlprogram, which can accurately discriminate the attribute of the imagedata irrespective of the heterogeneous region of the image data.

In accordance with one aspect of one or more embodiments of the presentinvention, there is provided an image attribute discrimination apparatusthat discriminates an attribute of image data based on a contentproduced by the image data, the image attribute discrimination apparatusincluding: a heterogeneous region specifying unit for specifying aheterogeneous region from the image data, the heterogeneous regionincluding a heterogeneous matter whose attribute is different from thatof the content originally produced by the image data; and an attributediscrimination unit for discriminating the attribute of the image databased on a feature quantity extracted from a pixel group except eachpixel in the heterogeneous region in each pixel of the image data.

According to the configuration, the heterogeneous region specifying unitspecifies the region including the heterogeneous matter (for example,the character string such as the telop) on the image data of theprocessing target, and the attribute discrimination unit discriminatesthe attribute of the image data based on the feature quantity that isobtained only from the pixel group of other region except the specifiedheterogeneous region.

Therefore, when the heterogeneous matter is included in the image dataof the processing target, the adverse effect of the feature quantityextracted from the heterogeneous region on the attribute discriminationprocessing can be removed. As a result, the attribute discriminationaccuracy can be improved.

The heterogeneous region specifying unit may specify a character regionincluding a character as the heterogeneous region.

According to the configuration, even if the text object (character)added later to the photograph is merged to the image data of theprocessing target, the heterogeneous region specifying unit determinesthat the later-added character is the heterogeneous matter, and theregion including the character is not used to extract the featurequantity. Accordingly, based on contents of the original image datalocated on the background of the character, the attribute can correctlybe discriminated even to the image data to which the character edit isadded.

The image attribute discrimination apparatus may further include arestoration unit for restoring a pixel identical to a target pixel withrespect to an out-of-target pixel region in each pixel of the imagedata, the out-of-target region that does not become a feature quantityextracting target.

According to the configuration, the restoration unit restores the pixelsin the background portion of the heterogeneous matter using informationon the pixels surrounding the character string such that the pixels inthe background portion become identical to the pixels surrounding thecharacter string. The background portion of the heterogeneous matter ishidden behind the heterogeneous matter (such as the character string).The attribute discrimination unit performs the attribute discriminationprocessing based on the restored image data. Therefore, the featurequantity in the portion hidden behind the character string caneffectively be utilized, and the accuracy of the attributediscrimination processing can be improved.

The image attribute discrimination apparatus may further include atarget pixel determination unit for determining whether each pixel inthe heterogeneous region specified by the heterogeneous regionspecifying unit is the target pixel whose feature quantity is extractedby the attribute discrimination unit, wherein the attributediscrimination unit discriminates the attribute of the image data basedon the feature quantity extracted from the pixel group except theout-of-target pixel in each pixel of the image data, the out-of-targetpixel being determined to be out of the target by the target pixeldetermination unit.

According to the configuration, the target pixel determination unitdetermines the target pixel and the out-of-target pixel in the pixels ofthe heterogeneous region specified by the heterogeneous regionspecifying unit. Accordingly, in extracting the feature quantity, theattribute discrimination unit does not refer to the pixel determined tobe the out-of-target pixel by the target pixel determination unit, butthe feature quantity is extracted only from the target pixel.

Therefore, because whether the pixel is the out-of-target pixel can moreparticularly be set in the heterogeneous region in extracting thefeature quantity, and a degree of freedom of design is enhanced inimplementing the image attribute discrimination apparatus thataccurately and efficiently performs the attribute discrimination.

The heterogeneous region specifying unit specifies the character regionincluding the character as the heterogeneous region, the image attributediscrimination apparatus further includes a character recognition unitfor recognizing the character in the character region specified by theheterogeneous region specifying unit, and the target pixel determinationunit preferably determines the pixel in the character region as thetarget pixel, when a degree of reliability of a character recognitionresult is not more than a predetermined value, the degree of reliabilityindicating likelihood that the character in the character region is thecharacter recognized by the character recognition unit.

Generally, the numerical value of the degree of reliability outputtedalong with the character recognition result is increased when thecharacter (string) is recognized more correctly, and the numerical valueof the degree of reliability is decreased when the character recognitionprocessing is performed while the matter that is not the character(string) is falsely extracted as the character string. Accordingly, onlywhen the degree of reliability is more than a predetermined value, theregion is determined to be the character region (heterogeneous region),and the pixels of the region are set to the out-of-target pixel. Thatis, the target pixel determination unit does not determine that thepixel having the low degree of reliability of the character recognitionresult is the out-of-target pixel even if the pixel having the lowdegree of reliability is initially determined to be the characterregion. Therefore, when the region that does not include the character(string) is falsely extracted as the character region, thefalsely-extracted region is not set to the out-of-target pixel, whichallows the target pixel determination unit to be prevented from failingto extract the feature quantity.

Therefore, the trouble that the accuracy of the attribute discriminationprocessing is degraded by expanding the out-of-target pixel in a blindway can be avoided.

The image attribute discrimination apparatus may further include akeyword extracting unit for extracting a keyword the character orcharacter string recognized by the character recognition unit; and aword association storage unit in which association between each keywordextracted by the keyword extracting unit and each attributediscriminated by the attribute discrimination unit is stored, whereinthe attribute discrimination unit refers to the word association storageunit, and the attribute discrimination unit discriminates the attributeof the image data in consideration of a level of association between thekeyword extracted from the character region of the image data and eachattribute.

According to the configuration, when the image data includes thecharacter region, the character recognition unit extracts the characterstring included in the character region, and the keyword extracting unitextracts at least one word (keyword) from the character string.

The attribute discrimination unit refers to the word association storageunit to recognize the association between the extracted keyword and theattribute, and the attribute discrimination unit considers the level ofthe association between the keyword and the attribute in discriminatingthe attribute of the image data.

Frequently the character string such as the telop included in the imagedata includes the character string that indicates the attribute of theimage data or has the deep association with the attribute. Accordingly,the attribute discrimination accuracy can be improved by settingcharacter string to one of the indexes of the attribute discriminationprocessing. For example, the attribute discrimination unit determinesthat the keyword “

” and the attribute “landscape” have the high association, and a highweight is added to the attribute “landscape” in discriminating theattribute of the image data when the keyword “

” is extracted.

The attribute discrimination unit checks the feature quantity of theimage data against a model feature quantity that is previously definedin each plurality of kinds of attributes, and the attributediscrimination unit discriminates the attribute of the image data bycomputing a degree of reliability of an attribute discrimination resultaccording to a degree of similarity between the feature quantity of theimage data and the model feature quantity, the degree of reliabilityindicating likelihood that the attribute of the image data is theattribute, and the association between the keyword and the attribute isstored in the word association storage unit as a score added to thedegree of reliability of the attribute discrimination result.

According to the configuration, the level of the association is storedas the score added to the degree of reliability of the attributediscrimination result. The attribute discrimination unit outputs thedegree of reliability with respect to each attribute that becomes acandidate in the image data, and the attribute discrimination unit addsthe score correlated to the keyword to the degree of reliability of thecorrelated attribute. The high score is added to the attribute havingthe high association with the keyword, thereby improving the degree ofreliability (likelihood that the image data is the attribute).Therefore, in consideration of the keyword included in the image data,the attribute discrimination unit can accurately discriminate theattribute of the image data based on the degree of reliability.

The image attribute discrimination apparatus may further include arestoration unit for restoring a pixel identical to a target pixel withrespect to an out-of-target pixel region in each pixel of the imagedata, the out-of-target region that does not become a feature quantityextracting target, wherein the restoration unit preferably performs therestoration when the degree of reliability of the attributediscrimination result is lower than a predetermined value.

According to the configuration of the image attribute discriminationapparatus, the restoration processing having a high processing load isomitted when the preferable result, that is, the high likelihood (degreeof reliability) of the discrimination result is obtained. Only when theattribute discrimination accuracy is degraded due to the low degree ofreliability, the restoration processing is performed in order to improvethe accuracy.

Accordingly, the balance between the improvement of the processingefficiency and the improvement of the attribute discrimination accuracycan be established.

The attribute discrimination unit may compute the degree of reliabilitylower with increasing region of the out-of-target pixel that does notbecome the feature quantity extracting target in each pixel of the imagedata.

The expanded region of the out-of-target pixel unit that a ratio of thepixel referred to in order to discriminate the attribute is decreased inone piece of image data, and possibly the attribute discriminationcannot precisely be performed compared with the case where all thepixels are referred to.

A user can be cautioned by underestimating the degree of reliability ofthe discrimination result outputted in such situations, or anothercountermeasure can be taken to improve the degree of reliability, whichcontributes to the improvement of the attribute discrimination accuracy.

The target pixel determination unit may determine each pixel in theheterogeneous region as the out-of-target pixel only when an areaoccupied by the heterogeneous region in the image data is more than apredetermined value.

According to the configuration, when the heterogeneous region specifiedby the heterogeneous region specifying unit is narrower (smaller) than apredetermined value, the target pixel determination unit does notperform the processing of determining that the pixel of theheterogeneous region is the out-of-target pixel.

Generally, for a small ratio of the area of the region including theheterogeneous matter to the area of the whole image data, the featurequantity obtained from the heterogeneous region has a small influence onthe attribute discrimination. In this case, there is a small advantagethat the discrimination accuracy is improved with respect to aprocessing time necessary to remove the heterogeneous region as theout-of-target region.

Therefore, as described above, the processing time can be shortenedwithout largely influencing the discrimination accuracy by providing therestriction that the out-of-target pixel is specified only when theratio of the heterogeneous region to the whole image data is not lowerthan the predetermined threshold.

The image attribute discrimination apparatus further includes a modelfeature quantity computing unit for computing a model feature quantityof a designated attribute using the feature quantity extracted from thepixel group except each pixel in the heterogeneous region specified bythe heterogeneous region specifying unit in each pixel of the imagedata, when image data and the designation of the attribute of the imagedata are inputted to the image attribute discrimination apparatus,wherein the attribute discrimination unit may check the feature quantityof the image data against the model feature quantity computed in eachattribute by the model feature quantity computing unit, and theattribute discrimination unit may discriminate the attribute of theimage data according to a degree of similarity between the featurequantity of the image data and the model feature quantity.

According to the configuration, the attribute discrimination unit checksthe feature quantity of the image data against the model featurequantity, and the attribute discrimination unit discriminates theattribute of the image data according to the degree of similarity of thechecking result. Accordingly, in order to perform the accurate attributediscrimination, it is necessary that the model feature quantity becorrectly defined according to each attribute.

Even if the image data including the heterogeneous matter (for example,the character string such as the telop) is captured as the image data ofthe learning target in the image attribute discrimination apparatus, themodel feature quantity computing unit uses the image data after removingthe heterogeneous matter specified by the heterogeneous regionspecifying unit, so that the model feature quantity can be produced morecorrectly. The attribute discrimination unit refers to the more correctmodel feature quantity, so that the attribute discrimination accuracycan be improved.

There is provided an attribute discrimination support apparatusaccording to one or more embodiments of the present invention thatdefines a model feature quantity in each attribute, an image attributediscrimination apparatus referring to the model feature quantity, theimage attribute discrimination apparatus discriminating an attribute ofimage data based on a content produced by the image data, the attributediscrimination support apparatus including: a heterogeneous regionspecifying unit for specifying a heterogeneous region from image datawhen the image data and designation of the attribute of the image dataare inputted, the heterogeneous region including a heterogeneous matterwhose attribute is different from that of the content originallyproduced by the image data; and a model feature quantity computing unitfor computing a model feature quantity of the designated attribute usinga feature quantity from a pixel group except each pixel in theheterogeneous region in each pixel of the image data.

According to the configuration, when the image data and the designationthe attribute of the image data are inputted in order to produce themodel feature quantity, the heterogeneous region specifying unitspecifies the heterogeneous region including the heterogeneous matterwhose attribute is different from that of the original image data. Then,the model feature quantity computing unit computes the model featurequantity of the designated attribute using the feature quantityextracted from the pixel group except each pixel in the heterogeneousregion.

Generally, the image attribute discrimination apparatus checks thefeature quantity of the image data against the model feature quantity,and the attribute discrimination unit discriminates the attribute of theimage data according to the degree of similarity of the checking result.Accordingly, in order to perform the accurate attribute discrimination,it is necessary that the model feature quantity be correctly definedaccording to each attribute.

Even if the image data including the heterogeneous matter (for example,the character string such as the telop) is captured as the image data ofthe learning target in the attribute discrimination support apparatus,the model feature quantity computing unit extracts and uses the featurequantity of the image data after removing the heterogeneous matterspecified by the heterogeneous region specifying unit, so that the modelfeature quantity can be produced more correctly. The attributediscrimination unit refers to the more correct model feature quantity,so that the attribute discrimination accuracy can be improved.

There is provided an image attribute discrimination method according toone or more embodiments of the present invention for discriminating anattribute of image data based on a content produced by the image data,the image attribute discrimination method including the steps of:specifying a heterogeneous region from the image data, the heterogeneousregion including a heterogeneous matter whose attribute is differentfrom that of the content originally produced by the image data; anddiscriminating the attribute of the image data based on a featurequantity extracted from a pixel group except each pixel in theheterogeneous region in each pixel of the image data.

There is provided a method for controlling an attribute discriminationsupport apparatus according to one or more embodiments of the presentinvention that defines a model feature quantity in each attribute, animage attribute discrimination apparatus referring to the model featurequantity, the image attribute discrimination apparatus discriminating anattribute of image data based on a content produced by the image data,the discrimination support apparatus controlling method including thesteps of: specifying a heterogeneous region from image data when theimage data and designation of the attribute of the image data areinputted, the heterogeneous region including a heterogeneous matterwhose attribute is different from that of the content originallyproduced by the image data; and computing a model feature quantity ofthe designated attribute using a feature quantity from a pixel groupexcept each pixel in the heterogeneous region in each pixel of the imagedata.

The image attribute discrimination apparatus and the attributediscrimination support apparatus may be implemented by a computer. Insuch cases, control programs of the image attribute discriminationapparatus or attribute discrimination support apparatus that cause thecomputer to implement the image attribute discrimination apparatus orthe attribute discrimination support apparatus by operating the computeras each of the units, and a computer-readable recording medium in whichthe control programs are recorded are also included in one or moreembodiments of the present invention.

There is provided an image attribute discrimination apparatus accordingto one or more embodiments of the present invention that discriminatesan attribute of image data based on a content produced by the imagedata, the image attribute discrimination apparatus including: aheterogeneous region specifying unit for specifying a heterogeneousregion from the image data, the heterogeneous region including aheterogeneous matter whose attribute is different from that of thecontent originally produced by the image data; and an attributediscrimination unit for discriminating the attribute of the image databased on a feature quantity extracted from a pixel group except eachpixel in the heterogeneous region in each pixel of the image data.

There is provided an attribute discrimination support apparatusaccording to one or more embodiments of the present invention thatdefines a model feature quantity in each attribute, an image attributediscrimination apparatus referring to the model feature quantity, theimage attribute discrimination apparatus discriminating an attribute ofimage data based on a content produced by the image data, the attributediscrimination support apparatus including: a heterogeneous regionspecifying unit for specifying a heterogeneous region from image datawhen the image data and designation of the attribute of the image dataare inputted, the heterogeneous region including a heterogeneous matterwhose attribute is different from that of the content originallyproduced by the image data; and a model feature quantity computing unitfor computing a model feature quantity of the designated attribute usinga feature quantity from a pixel group except each pixel in theheterogeneous region in each pixel of the image data.

There is provided an image attribute discrimination method according toone or more embodiments of the present invention for discriminating anattribute of image data based on a content produced by the image data,the image attribute discrimination method including the steps of:specifying a heterogeneous region from the image data, the heterogeneousregion including a heterogeneous matter whose attribute is differentfrom that of the content originally produced by the image data; anddiscriminating the attribute of the image data based on a featurequantity extracted from a pixel group except each pixel in theheterogeneous region in each pixel of the image data.

There is provided a method for controlling an attribute discriminationsupport apparatus according to one or more embodiments of the presentinvention that defines a model feature quantity in each attribute, animage attribute discrimination apparatus referring to the model featurequantity, the image attribute discrimination apparatus discriminating anattribute of image data based on a content produced by the image data,the discrimination support apparatus controlling method including thesteps of: specifying a heterogeneous region from image data when theimage data and designation of the attribute of the image data areinputted, the heterogeneous region including a heterogeneous matterwhose attribute is different from that of the content originallyproduced by the image data; and computing a model feature quantity ofthe designated attribute using a feature quantity from a pixel groupexcept each pixel in the heterogeneous region in each pixel of the imagedata.

Accordingly, advantageously the attribute of the image data canaccurately be discriminated irrespective of the heterogeneous region ofthe image data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration of a main part of adigital photo frame according to one or more embodiments of the presentinvention;

FIG. 2 is a view showing an appearance of the digital photo frame of theembodiment;

FIG. 3A is a view showing an example of original image data(Fujiyama.jpg) of a processing target, and FIG. 3B is a view showing aspecific example of a region extracted as a character region from theimage data of FIG. 3A;

FIG. 4A is a view showing a specific example of a color histogram thatis produced from a target pixel except the character region of FIG. 3Bin the image data (Fujiyama.jpg) of FIG. 3A, FIG. 4B is a view showing aspecific example of a histogram that is produced from the image data ofFIG. 3A while the character region is not excluded, that is, a histogramthat is produced without applying one or more embodiments the presentinvention, and FIG. 4C is a view showing a specific example of a modelhistogram that is a previously-stored model feature quantity;

FIG. 5 is a view showing an example of the image data after restorationprocessing is performed to a heterogeneous region;

FIG. 6 is a flowchart showing a scene discrimination processing flow ofthe digital photo frame of the embodiment of the present invention;

FIG. 7 is a view showing specific examples of a character recognitionresult and a degree of reliability thereof, which are outputted by acharacter recognition unit of the digital photo frame;

FIG. 8 is a view showing specific examples of a scene discriminationresult and a degree of reliability thereof, which are outputted by ascene discrimination unit of the digital photo frame;

FIG. 9 is a view showing a specific example a correspondence tableexpressing an association between a keyword and a scene, which is storedin a character score storage unit of the digital photo frame;

FIG. 10A is a flowchart showing an example of the scene discriminationprocessing flow of the digital photo frame of the embodiment of thepresent invention;

FIG. 10B is a flowchart showing an example of the scene discriminationprocessing flow of the digital photo frame of the embodiment of thepresent invention;

FIG. 11 is a view showing another specific example of the regionextracted as the character region from the image data of FIG. 3A;

FIG. 12 is a view showing still another specific example of the regionextracted as the character region from the image data of FIG. 3A;

FIG. 13 is a block diagram showing a configuration of a main part of adigital photo frame according to another embodiment of the presentinvention;

FIG. 14 is a flowchart showing an example of a scene learning processingflow of the digital photo frame of another embodiment of the presentinvention;

FIG. 15 is a block diagram showing a configuration of a main part of anattribute discrimination support apparatus according to anotherembodiment of the present invention;

FIG. 16 is a view showing an example of image data including a smear asa heterogeneous region;

FIG. 17 is a view showing an example of image data including a regionwhere an object as the heterogeneous region is merged in an image editprocess;

FIG. 18 is a view showing an example of image data including a foldedportion as the heterogeneous region because an original is scanned whilean end of the original is folded; and

FIG. 19 is a view showing an example of image data including a fingershadow taken in a photograph as the heterogeneous region.

DETAILED DESCRIPTION First Embodiment

Hereinafter, embodiments of the present invention will be described withreference to the drawings.

The case, in which an image attribute discrimination apparatus accordingto one or more embodiments of the present invention is applied to adigital photo frame that is an image display apparatus displaying imagedata, will be described below by way of example. A digital photo frameaccording to a first embodiment of the present invention can correctdisplay data of image data to display the image data on a display unitof the image attribute discrimination apparatus according to a scene(attribute) of the image data discriminated by the image attributediscrimination apparatus. The image attribute discrimination apparatusof one or more embodiments of the present invention is not limited tothe digital photo frame, but the image attribute discriminationapparatus can suitably be applied to various display devices, such as adigital camera, a digital video camera, a digital video recorder/player,a digital television, a personal computer, a printer, and a scanner,which can perform different processing to the image data in eachdiscriminated scene.

[Appearance of Digital Photo Frame]

FIG. 2 is a view showing an appearance of the digital photo frame 100 ofthe first embodiment of the present invention. The digital photo frame100 reads the image data and outputs the image data as display data to adisplay unit 3 to display an image. The image data such as one or aplurality of still images and moving images is stored in the digitalphoto frame 100 or recorded in a removable external recording medium.For example, the digital photo frame 100 displays a photograph (imagedata) taken with a digital camera on the display unit 3 to act as apicture frame. The digital photo frame 100 can sequentially display theplurality of photographs like slide show, display a plurality of piecesof image data at once with an arbitrary layout in arbitrary timing, anddisplay a list of a large amount of image data in a thumbnail form.

A manipulation unit 4 of the digital photo frame 100 may be implementedby a button or a touch panel, which is provided in a main body of thedigital photo frame 100, or use of a remote controller as shown in FIG.2.

In the first embodiment, in order to improve the display of the imagedata, a digital photo frame 100 has a function (hereinafter referred toas scene-by-scene automatic correction function) of correcting the imagedata scene by scene to display the image. Various setting manipulationsrelating to functions of the digital photo frame 100 can be realized bythe use of the manipulation unit 4.

For example, as shown in FIG. 2, the digital photo frame 100 may displaya manipulation screen on the display unit 3 in order to cause a user toset whether the scene-by-scene automatic correction function is enabled.When enabling the scene-by-scene automatic correction function, the usermanipulates an arrow key or an enter button provided in the manipulationunit 4 to select an option enabling the scene-by-scene automaticcorrection function.

[Configuration of Digital Photo Frame]

FIG. 1 is a block diagram showing a configuration of a main part of thedigital photo frame 100 of the first embodiment of the presentinvention.

As shown in FIG. 1, the digital photo frame 100 of the first embodimentincludes a control unit 1, a storage unit 2, the display unit 3, themanipulation unit 4, a temporary storage unit 5, and a bus 6. The bus 6is a common signal line through which the data is transmitted andreceived among the units of the digital photo frame 100.

The control unit 1 performs various arithmetic operations by executing aprogram that is read from the storage unit 2 to the temporary storageunit 5, and wholly controls the units of the digital photo frame 100through the bus 6. The control unit 1 includes at least a characterregion extracting unit 11 and a scene discrimination unit 13 asfunctional blocks. The control unit 1 may include a target pixelspecifying unit 12, a character recognition unit 14, and a restorationunit 15 as functional blocks. In the digital photo frame 100, thefunctional block 11 to 15 acts as an image attribute discriminationapparatus 10 that performs a scene discrimination function. In the firstembodiment, the control unit 1 includes a scene-by-scene processingperforming unit 16 as a functional block. In the digital photo frame100, the functional block 16 acts as an image processing apparatus thatperforms the scene-by-scene automatic correction function.

Each functional block of the control unit 1 can be realized, such that aCPU (central processing unit) reads a program stored in the storage unit2 implemented by a ROM (Read Only Memory) to the temporary storage unit5 implemented by a RAM (Random Access Memory) and executes the program.

A control program and an OS program, which are executed by the controlunit 1, and various pieces of fixed data, which are read when thecontrol unit 1 performs various functions (such as the scenediscrimination function and the scene-by-scene automatic correctionfunction of one or more embodiments of the present invention) possessedby the digital photo frame 100, are stored in the storage unit 2. In thefirst embodiment, for example, the storage unit 2 includes an image datastorage unit 30, a scene feature quantity storage unit 31, a characterchecking dictionary storage unit 32, and a character score storage unit33. Various pieces of fixed data are stored in the storage unit 2. Forexample, the storage unit 2 is implemented by a nonvolatile memory suchas an EEPROM (Electrically EPROM) and a flash memory, in which contentsare rewritable. A storage unit (such as the character checkingdictionary storage unit 32 and the character score storage unit 33), inwhich information whose contents are not necessary to be rewritten isstored, may be implemented by a ROM (Read Only Memory, not shown) thatis a read-only semiconductor memory separated from the storage unit 2.

The image data that is a target processed by the digital photo frame 100as the image processing apparatus is stored in the image data storageunit 30. A feature quantity of a scene, which becomes a model referredto by the scene discrimination unit 13 when the scene discriminationunit 13 discriminates the scene of the image data, is stored in thescene feature quantity storage unit 31. Information on a character,which is referred to by the character recognition unit 14 when thecharacter recognition unit 14 recognizes a character included in theimage data, that is, a character checking dictionary is stored in thecharacter checking dictionary storage unit 32, when the control unit 1includes the character recognition unit 14. Score information is storedin the character score storage unit 33. The score information expressesassociation between a character (string) recognized by the characterrecognition unit 14 and a discriminated scene in the form of a numericalvalue (score).

As described above with reference to FIG. 2, the display unit 3 displaysthe image data captured from the image data storage unit 30 or from theexternal recording medium through an external interface (not shown), ordisplays a manipulation screen as a GUI (Graphical User Interface)screen on which the user manipulates the digital photo frame 100. Forexample, the display unit 3 includes a display device such as an LCD(Liquid Crystal Display) and an organic EL display.

The manipulation unit 4 is used when the user inputs an instructionsignal to the digital photo frame 100 to manipulate the digital photoframe 100. As described above, in the first embodiment, the manipulationunit 4 is formed as the remote controller. When a button (such as thearrow key, the enter key, and a character input key) provided in themanipulation unit 4 is pressed down, a corresponding signal is outputtedas an infrared signal from a light emitting portion of the manipulationunit 4, and the signal is inputted to the digital photo frame 100through a light receiving portion provided in the main body of thedigital photo frame 100.

The temporary storage unit 5 is a so-called working memory in which dataused in an arithmetic operation and an arithmetic result are temporarilystored in processes of various pieces of processing performed by thedigital photo frame 100. The temporary storage unit 5 is implemented bya RAM (Random Access Memory). More specifically, the control unit 1expands the image data that becomes a scene discrimination processingtarget in an image memory 5 a of the temporary storage unit 5, andanalyzes the image data in detail in units of pixels.

The character region extracting unit 11 of the control unit 1 extracts aheterogeneous region included in the image data of the processingtarget. In the first embodiment, particularly the character regionextracting unit 11 extracts a character region including a character(string) such as an alphanumeric character, a hiragana character, akatakana character, a kanji character, and a symbol as the heterogeneousregion.

FIG. 3A is a view showing an example of the original image data of theprocessing target. Although image data (file name: Fujiyama.jpg) shownin FIG. 3A is a photograph in which a landscape is originally taken, atext object is added to the landscape photograph in an image editprocess in the image data. The character region extracting unit 11specifies a character-like shape from a color difference withsurrounding pixels and a texture, and the character region extractingunit 11 extracts a region including the character-like shape as thecharacter region. FIG. 3B is a view showing an example of the regionextracted as the character region from the image data of FIG. 3A. In thefirst embodiment, for example, as shown in FIG. 3B, the character regionextracting unit 11 extracts a circumscribed rectangle having acharacter-string-like shape in a comprehensive way as the characterregion. In the example shown in FIG. 3B, the character region extractingunit 11 extracts a circumscribed rectangle of a character string “[

]” included in the original image data as a first character region Ar1,a circumscribed rectangle of a character string “

” as a second character region Ar2, and a circumscribed rectangle of acharacter string “

” as a third character region Ar3.

A well-known technique is appropriately adopted as the character regionextracting technique (for example, see the technique disclosed inMasatoshi Okutomi, et al., “Digital Image Processing”, CG-ARTS SocietyPress, Mar. 1, 2007 (2nd edition, 2nd print), P. 208 to 210, Section11-1 “Region Feature Quantity”.

The target pixel specifying unit 12 specifies whether each pixel of theheterogeneous region extracted by the character region extracting unit11 becomes an analysis target of the scene discrimination processing.Specifically, the target pixel specifying unit 12 fixes a flag in eachpixel. The flag indicates whether the pixel becomes the analysis targetor the pixel out of the target. For example, the target pixel specifyingunit 12 sets a flag “TRUE” indicating the analysis target to the pixelthat becomes the analysis target, and the target pixel specifying unit12 sets a flag “FALSE” indicating the pixel out of the analysis targetto the pixel that does not become the analysis target. Therefore, thepixel that becomes the feature quantity analysis target is specifiedfrom the image data in the scene discrimination processing.

The target pixel specifying unit 12 may specify all the pixels in allthe heterogeneous regions (character regions) extracted by the characterregion extracting unit 11 as the pixel out of the analysis target. Thatis, the three character regions shown in FIG. 3B of the first characterregion Ar1, second character region Ar2, and third character region Ar3may be specified as the pixel out of the analysis target. Alternatively,based on a predetermined condition, the target pixel specifying unit 12may specify only the pixel satisfying the condition in the heterogeneousregions extracted by the character region extracting unit 11 as thetarget pixel or the pixel out of the analysis target.

The scene discrimination unit 13 discriminates the scene of the imagedata. Particularly the scene discrimination unit 13 performs theanalysis and the extraction of the feature quantity only to the pixelspecified as the target pixel by the target pixel specifying unit 12 inall the pixels of the image data. The scene discrimination unit 13discriminates the scene of the image data by checking the extractedfeature quantity against a model feature quantity that is previouslystored every scene in the scene feature quantity storage unit 31. In thefirst embodiment, for example, the scene discrimination unit 13 analyzesa pixel value for a region except the three character regions shown inFIG. 3B of the first character region Ar1, second character region Ar2,and third character region Ar3, and the scene discrimination unit 13extracts the feature quantity.

In the first embodiment, the scene discrimination unit 13 produces ahistogram of the target pixel as the whole feature quantity of the imagedata based on the color or texture of the target pixel. The scenediscrimination unit 13 compares and checks the model feature quantity ineach scene and the feature quantity extracted from the image data, anddiscriminates the scene of the most similar model feature quantity asthe scene of the image data. The scene discrimination processing will bedescribed in detail below.

FIG. 4A is a view showing a specific example of a color histogram thatis produced by the scene discrimination unit 13 from the target pixelexcept the character region of FIG. 3B in the image data (Fujiyama.jpg)of FIG. 3A.

FIG. 4B is a view showing a specific example of a histogram that isproduced from the image data of FIG. 3A while the character region isnot excluded, that is, a histogram that is produced without applying oneor more embodiments of the present invention,

FIG. 4C is a view showing a specific example of a model histogram thatis a model feature quantity stored in the scene feature quantity storageunit 31. For example, it is assumed that standard model histograms arepreviously stored in the scene feature quantity storage unit 31 withrespect to 11 kinds of scenes of (1) person, (2) landscape, (3) nightview, (4) sunset, (5) firework, (6) room interior, (7) snow, (8) beach(9) flower, (10) cooking, and (11) business card and document. The modelhistogram of FIG. 4C shows a specific example of the model histogram of“(2) landscape”. In the histograms shown in FIGS. 4A to 4C, a horizontalaxis indicates a bin corresponding to each color, and a vertical axisindicates frequency (the number of pixels)×edge intensity.

It is assumed that the text objects “[

]”, “

3,776 m”, and “

”, which are included in the image data (Fujiyama.jpg) shown in FIG. 3A,include only yellow pixels.

The histogram of one or more embodiments of the present invention ofFIG. 4A differs from the histogram of FIG. 4B in that it does notinclude yellow color intensity as the feature. This is because thetarget pixel specifying unit 12 excludes the character region includingthe yellow character string from the target pixel.

When the histogram of FIG. 4B is used while “yellow color intensity” ismixed, unfortunately a determination that the histogram of FIG. 4B isnot similar to the model histogram of FIG. 4C is made, or unfortunatelya reliable discrimination result is not obtained because the histogramof FIG. 4B has a low degree of similarity even if the determination thatthe histogram of FIG. 4B is similar to the model histogram of FIG. 4C ismade.

On the other hand, according to the image attribute discriminationapparatus 10 of one or more embodiments of the present invention,because the feature quantity of “yellow color intensity” that isdifferent from the original scene (attribute) can be excluded, the scenediscrimination unit 13 determines that the degree of similarity becomesthe highest between the histogram (FIG. 4A) obtained from the image data(Fujiyama.jpg) and the model histogram (FIG. 4C) of “(2) landscape”, andthe scene discrimination unit 13 can correctly discriminate the scene ofthe image data (Fujiyama.jpg) as “(2) landscape”.

The scene discrimination unit 13 quantifies likelihood (probability thatthe image data is the scene) that the image data is the scene as a“degree of reliability” according to the degree of similarity betweenthe feature quantity of the image data of the processing target and themodel feature quantity, and the scene discrimination unit 13 may outputthe degree of reliability along with the discrimination result.

The degree of reliability of the scene discrimination result isincreased as the feature quantity (histogram) extracted from the imagedata is similar to the model feature quantity stored in the scenefeature quantity storage unit 31.

A well-known technique is appropriately adopted as the feature quantityextracting technique (for example, see the technique disclosed inMasatoshi Okutomi, et al., “Digital Image Processing”, CG-ARTS SocietyPress, Mar. 1, 2007 (2nd edition, 2nd print), P. 208 to 210, Section11-1 “Region Feature Quantity”.

According to the configuration, in discriminating the scene of the imagedata, the character region extracting unit 11 extracts the heterogeneousregion (for example, the text object with respect to the landscapephotograph) when the image data includes the heterogeneous region. Thenthe target pixel specifying unit 12 specifies the pixel that is excludedfrom the analysis target for the purpose of the scene discrimination inthe pixels of the heterogeneous region. Finally the scene discriminationunit 13 extracts the feature quantity from the pixel that is specifiedas the analysis target in all the pixels of the image data, anddiscriminates the scene of the image data based on the extracted featurequantity.

For the image data including the heterogeneous region different from theoriginal scene, only the region except the heterogeneous region isanalyzed to perform the scene discrimination. Therefore, advantageouslythe scene of the image data can accurately be discriminated irrespectiveof the heterogeneous region of the image data.

As described above, the control unit 1 may further include the characterrecognition unit 14 and the restoration unit 15.

The character recognition unit 14 recognizes the character (string)included in the character region when the heterogeneous region extractedby the character region extracting unit 11 is the character region thatpossibly includes the character (string). The character recognition unit14 compares a model shape of any character stored in the characterchecking dictionary storage unit 32 and a character (-like) shape thatis included in the character region and specified from the color ortexture, and the character recognition unit 14 specifies the character(string) included in the character region.

In the example shown in FIG. 3B, the character recognition unit 14recognizes the character string “[

]” from the first character region Ar1 extracted by the character regionextracting unit 11, recognizes the character string “

3,776 m” from the second character region Ar2, recognizes the characterstring “

” from the third character region Ar3, and outputs the character stringsas the text data. The output text data is probably a word associatedwith the content of the image data. Therefore, the scene discriminationunit 13 may refer to the character score storage unit 33 to discriminatethe scene of the image data in consideration of the association betweenthe scene and the meaning of the word of the text data.

The character recognition unit 14 may quantify likelihood that thecharacter (string) included in the character region is the recognizedcharacter (string) as the “degree of reliability” according to thedegree of similarity of the model shape of the character, and thecharacter recognition unit 14 may output the degree of reliability alongwith the recognition result. When the character recognition result hasthe low degree of reliability, the target pixel specifying unit 12determines that something looks like the character included in thecharacter region is actually not the character (that is, not theheterogeneous region), and the target pixel specifying unit 12 can takeinto account the fact in specifying the target pixel.

A well-known technique is appropriately adopted as the characterrecognition technique (for example, see the technique disclosed inMasatoshi Okutomi, et al., “Digital Image Processing”, CG-ARTS SocietyPress, Mar. 1, 2007 (2nd edition, 2nd print), P. 208 to 210, Section11-1 “Region Feature Quantity”.

In order to set the pixel (heterogeneous region) that is specified asthe pixel out of the analysis target by the target pixel specifying unit12 to the feature quantity extracting target, the restoration unit 15restores the pixel out of the analysis target such that the pixel out ofthe analysis target has a pixel value identical to that of the pixels ofthe analysis target based on the pixel value of the surrounding pixel ofthe analysis target.

FIG. 5 is a view showing an example of the image data in whichrestoration processing is performed to the heterogeneous region. Asshown in FIG. 5, the restoration unit 15 performs restoration processingto the first character region Ar1, second character region Ar2, andthird character region Ar3, which are extracted as the heterogeneouscharacter region in FIG. 3B, whereby the pixels whose attributes areidentical (color, texture and the like match the landscape of MountFuji) to those of pixel in the region except the character regions Ar1,Ar2, and Ar3 are interpolated in the character regions Ar1, Ar2, andAr3. The scene discrimination unit 13 refers to the restored restorationregions Ar1′ to Ar3′ for the purpose of the scene discrimination.

The target pixel specifying unit 12 re-specifies the pixels restored bythe restoration unit 15 as the target pixel, the scene discriminationunit 13 performs the analysis and the extraction of the feature quantityto the newly-specified target pixel (restored pixel). Therefore, theregion that is taken into account for the purpose of the scenediscrimination can be widened with for the one piece of pixel data, andthe accuracy of the scene discrimination result can be enhanced.

A well-known technique is appropriately adopted as the restorationtechnique (for example, see the technique disclosed in Toshiyuki Amano,et al., “Image Interpolation by BPLP using Eigen Space Method”, IEICETransaction, Vol. J85-D-II, No. 3, P. 457 to 465.

The scene-by-scene processing performing unit 16 performs differentprocessing to the image data discriminated by the scene discriminationunit 13 scene by scene. In the first embodiment, the scene-by-sceneprocessing performing unit 16 corrects the display data of the imagedata such that each discriminated scene is displayed in the mostbeautiful state. That is, the scene-by-scene processing performing unit16 acts as an image data correction unit that performs thescene-by-scene automatic correction function in the digital photo frame100.

Therefore, the digital photo frame 100 can be configured as the imageprocessing apparatus that can beautifully display the scene of the imagedata while the scene is always corrected to the optimum state.

Specifically, the scene-by-scene processing performing unit 16 correctsthe image data discriminated to be the scene of “firework” or “flower”in a brightly-colored manner by increasing a chroma of the display data,performs the correction in which a hue is slightly changed to highlightred to the image data discriminated to be the scene of “sunset”, andperforms the correction in which lightness is increased to create apositive atmosphere to the image data discriminated to be the scene of“room interior (event, party)”.

[Scene Discrimination Processing Flow]

FIG. 6 is a flowchart showing a scene discrimination processing flow ofthe digital photo frame 100 of the first embodiment.

The image attribute discrimination apparatus 10 expands the image data(for example, “Fujiyama.jpg” shown in FIG. 3A) that becomes the scenediscrimination processing target from the image data storage unit 30 inthe image memory 5 a of the temporary storage unit 5 (S101). The targetpixel specifying unit 12 defines a flag array of each pixel of theexpanded image data and initializes the flag array (S102). For example,when the image data includes the pixels of a width (width of imagedata)×height (height of image data)=x pixels×y pixels, the target pixelspecifying unit 12 defines a feature quantity extracting target flagarray feat_use_flag[x][y]. In this case, when the heterogeneous regionis not included, the target pixel specifying unit 12 initializes theflags of all the pixels to TRUE because basically the target pixelspecifying unit 12 sets all the pixels of the image data to featurequantity extracting target. As described above, when the flag is “TRUE”,the pixel is the feature quantity extracting target.

Then the character region extracting unit 11 extracts the characterregion as the heterogeneous region with respect to the image dataexpanded in the image memory 5 a (S103). As shown in FIG. 3B, thecharacter region extracting unit 11 extracts the three character regionsof the first character region Ar1 to the third character region Ar3.

The target pixel specifying unit 12 specifies whether each pixelbelonging to the extracted character region becomes the pixel of thefeature quantity extracting target for the scene discrimination (S104).In the first embodiment, because the flags of all the pixels are set to“TRUE” by the initialization, the flag is set to “FALSE” with respect tothe pixels in the three character regions. As described above, when theflag is “FALSE”, the pixel is the pixel out of the feature quantityextracting target.

The scene discrimination unit 13 extracts the feature quantity only fromthe pixel satisfying the condition that the flag is “TRUE” in the pixelsof the image data (S105). Specifically, the scene discrimination unit 13produces a color histogram. The scene discrimination unit 13 checks thehistogram produced in S105 against the model histogram of each scenestored in the scene feature quantity storage unit 31, therebydiscriminating the scene of the image data (S106). For example, whendetermining that the degree of similarity becomes the maximum betweenthe histogram (for example, FIG. 4A) obtained from the image data(Fujiyama.jpg) of the processing target and the model histogram (forexample, FIG. 4C) of “(2) landscape” stored in the scene featurequantity storage unit 31, the scene discrimination unit 13 discriminatesthat the scene of the image data is “(2) landscape”.

Finally the scene discrimination unit 13 outputs the scenediscrimination result “(2) landscape” to the scene-by-scene processingperforming unit 16 (S107).

Therefore, for example, the scene-by-scene processing performing unit 16can correct the display data of the image data of FIG. 3A to output thedisplay data to the display unit 3 such that the photograph of thelandscape is displayed most beautiful based on the scene discriminationresult “(2) landscape”.

According to the configuration, in discriminating the scene of the imagedata, the character region extracting unit 11 extracts the heterogeneousregion (such as the text object with respect to the landscapephotograph) when the heterogeneous region is included in the image data.Then, for the purpose of the scene discrimination, the target pixelspecifying unit 12 specifies the pixel excluded from the analysis targetwith respect to the pixels in the heterogeneous region. Finally, thescene discrimination unit 13 extracts the feature quantity of the imagedata except the pixel excluded from the analysis target in all thepixels of the image data, and the scene discrimination unit 13 performsthe scene discrimination of the image data based on the extractedfeature quantity.

For the image data including the heterogeneous region different from theoriginal scene, only the region except the heterogeneous region isanalyzed to perform the scene discrimination. Therefore, advantageouslythe scene of the image data can accurately be discriminated irrespectiveof the heterogeneous region of the image data.

In the above example, the text objects of “[

]”, “

3,776 m”, and “

” are added to the photograph of Mount Fuji. For example, when the textobjects are formed by the yellow pixels, the heterogeneous region havingthe color and texture that are not included in the usual landscapephotograph of Mount Fuji is included in the image data. When thehistogram is produced with respect to all the pixels while such aheterogeneous region is included, the histogram includes the shape thatis not usually included in the original landscape scene (for example,FIG. 4B). When the matching is performed based on such a histogram,unfortunately the photograph is falsely discriminated to be anotherscene, or unfortunately only the discrimination result having theextremely low degree of reliability is obtained even if the photographis discriminated to be the photograph of Mount Fuji.

On the other hand, in one or more embodiments of the present invention,the histogram is produced with respect to only the pixels (that is, thepixels constituting the landscape photograph of Mount Fuji) in theregion except the yellow region, and the matching is performed based onthe histogram. The scene discrimination processing can be performedwhile an adverse effect of “feature of strong yellow” that does notexpress the scene is prevented.

Second Embodiment

An additional configuration that more efficiently performs the scenediscrimination processing while the configuration of the firstembodiment is used as the basic configuration or an additionalconfiguration that further improves the accuracy of the scenediscrimination processing will be described in a second embodiment.

[Consideration of Area of Character Region]

The target pixel specifying unit 12 may determine whether the pixelbecomes the analysis target in the scene discrimination processingaccording to a ratio of the character region extracted by the characterregion extracting unit 11 to the whole image data. Specifically, when anarea ratio of the character region is not lower than a predeterminedthreshold, it is considered that the character region has a largeinfluence on the scene discrimination processing of the image data, andthe target pixel specifying unit 12 excludes the character region fromthe analysis target pixel (flag is set to FALSE). On the other hand,when an area ratio of the character region is lower than a predeterminedthreshold, it is considered that the character region has a smallinfluence on the scene discrimination processing of the image data evenif the character region is different from the original scene, and thetarget pixel specifying unit 12 leaves the flags of all the pixels ofthe image data TRUE.

According to the configuration, for the small adverse effect of theheterogeneous region, the target pixel specifying processing (processingof setting flag to FALSE/TRUE) can be omitted in target pixel specifyingunit 12, so that the scene discrimination processing can efficiently beperformed while the accuracy of the scene discrimination processing ismaintained.

[Consideration of Likelihood that Character Region is Character]

Alternatively, when the character recognition unit 14 performs thecharacter recognition processing to the character region, the targetpixel specifying unit 12 may determine whether the pixel becomes theanalysis target in the scene discrimination processing according to thedegree of reliability of the character recognition result. FIG. 7 is aview showing specific examples of the character recognition result andthe degree of reliability thereof, which are outputted by the characterrecognition unit 14. FIG. 7 shows an example of the result, in which thecharacter recognition unit 14 performs the character recognition in eachof the three character regions extracted from the image data(Fujiyama.jpg) by the character region extracting unit 11 and outputsthe degree of reliability with respect to the character recognitionresult of each region.

Referring to the specific example of FIG. 7, when the degree ofreliability is not lower than a predetermined threshold, the targetpixel specifying unit 12 determines that probably the character (string)is included as the heterogeneous matter in the character region, and thetarget pixel specifying unit 12 sets the flags of all the pixels in thecharacter region to FALSE. On the other hand, when the degree ofreliability is lower than the predetermined threshold, the target pixelspecifying unit 12 determines that the character (string) is actuallynot included in the region regarded as the character region, and leavesthe flag of the pixel in the region TRUE. In the example shown in FIG.7, when a score of “55” is the threshold of the degree of reliability,because the character recognition results of the three character regionsof FIG. 3B have the scores of 55 or more, the target pixel specifyingunit 12 sets the flags of all the pixels in the character region toFALSE.

According to the configuration, the target pixel specifying unit 12re-determines that the region, which is not recognized as the characterwith specific likelihood by the character recognition unit 14 while oncedetermined to be the character region by the character region extractingunit 11, is actually not the heterogeneous region including theheterogeneous matter. The target pixel specifying unit 12 then specifiesthe region as the analysis target in the scene discriminationprocessing. Therefore, the region falsely recognized as the characterregion can be prevented from being excluded from the analysis target, sothat the accuracy of the scene discrimination processing can beimproved.

[Output of Degree of Reliability of Scene Discrimination Result]

In performing the matching between the histogram of the image data ofthe processing target and the model histogram stored in the scenefeature quantity storage unit 31, the scene discrimination unit 13 mayoutput the degree of reliability of the scene discrimination resultaccording to the degree of similarity. FIG. 8 is a view showing specificexamples of a scene discrimination result and a degree of reliabilitythereof, which are outputted by the scene discrimination unit 13. In theexample shown in FIG. 8, as a result of the matching performed by thescene discrimination unit 13, the histogram of the image data of FIG. 3Bhas the highest degree of similarity to the model histogram of the scene“landscape” and the degree of reliability has the score of “60”. Then,the degree of reliability is computed according to the degree ofsimilarity to the model histogram in the order of “beach”, “snow”,“cooking”, and “night view”.

The scene discrimination unit 13 discriminates the scene of the imagedata as the “landscape”, and the scene discrimination unit 13 outputsthe discrimination result to the scene-by-scene processing performingunit 16 along with the score of “60” of the degree of reliability.Alternatively, the scene discrimination unit 13 may perform anotherpiece of processing when the degree of reliability is lower than thepredetermined threshold.

For example, the scene discrimination unit 13 displays a message thatthe scene of the image data cannot be discriminated to the user oroutputs the message to the scene-by-scene processing performing unit 16.In such cases, the scene-by-scene processing performing unit 16 mayperform not scene-by-scene processing but default processing to theimage data. Alternatively, the scene discrimination unit 13 clearlydisplays a message of the low degree of reliability to the user. In suchcases, the user confirms the discrimination result, and the user canperform the correction when an error exists. According to theconfiguration, when the scene discrimination result is incorrect, theuser can be prevented from overlooking the incorrect scenediscrimination result.

The scene discrimination unit 13 may output the degree of reliability ofthe scene discrimination result while adding an area of the region outof the analysis target as the character region to the whole image datato the degree of reliability. Specifically, the number of pixels thatbecome the analysis targets is decreased with increasing area of thecharacter region, and the accuracy of the scene discrimination of theimage data is degraded. Therefore, the scene discrimination unit 13 mayadjust the score such that the degree of reliability of each scene shownin FIG. 8 is decreased with increasing area of the character region inthe image data.

Therefore, the image attribute discrimination apparatus 10 can moreprecisely understand the degree of reliability of the scenediscrimination result to perform a correct step corresponding to thedegree of reliability.

[Restoration of Character Region]

For example, when the degree of reliability of the scene discriminationresult is lower than a specific value, the restoration unit 15 performsthe restoration processing to the heterogeneous region excluded from theanalysis target, and the restoration unit 15 may perform the scenediscrimination processing to the image data again while adding the pixelvalue of the restored region. When the number of pixels of the analysistarget is increased, the scene discrimination processing can beperformed with higher reliability. In the restoration region, thelikelihood that the pixel value of the post-restoration is identical tothe original pixel value is lower than that of the region of the targetpixel to which the restoration is not performed. Therefore, when thehistogram is produced, the scene discrimination unit 13 may add a weightto the feature quantity extracted from the restored pixel such that thefeature quantity is multiplied by a coefficient of 0 to 1.

Irrespective of the degree of reliability, the restoration processingperformed by the restoration unit 15 may be configured to be alwaysperformed to the pixel that is specified as the pixel out of theanalysis target by the target pixel specifying unit 12. However, in theconfiguration, the restoration processing is performed while focusing onthe low degree of reliability. Therefore, preferably an opportunity ofthe high-load restoration processing can be reduced to improve theprocessing efficiency of the whole of the image attribute discriminationapparatus 10.

[Consideration of Semantic Content of Character (String)]

In the image attribute discrimination apparatus 10 of the secondembodiment, the character recognition result (for example, the characterstring shown in FIG. 7) may be used as one of indexes of the scenediscrimination processing. That is, the degree of reliability may becomputed in each scene of the scene discrimination result shown in FIG.8 in consideration of what meaning of the word included in therecognized character string.

For example, the image attribute discrimination apparatus 10 alsoincludes a keyword extracting unit 17 (shown in FIG. 1). The storageunit 2 includes the character score storage unit 33. The keywordextracting unit 17 extracts a keyword as a minimum unit of thecharacters (string) having the meaning by performing a morphologicalanalysis to the character string that is recognized in each region bythe character recognition unit 14. The keyword extracting unit 17 may beconfigured in any way. For example, the keyword extracting unit 17 maybe configured to extract a substantive keyword from the characterstring. A correspondence table is stored in the character score storageunit 33. The correspondence table shows how many points of the degree ofreliability are added to the scene in each keyword. That is, in thecharacter score storage unit 33, the association between the keyword andthe scene (attribute) is stored as the score that should be added to thedegree of reliability.

In the example shown in FIG. 7, the keyword extracting unit 17 extractsthe keywords “

” and “

” from the character string “[

]” of the first character region Ar1. Similarly the keyword extractingunit 17 extracts the keyword from the character string of the remainingregions.

Based on the keyword extracted by the keyword extracting unit 17, thescene discrimination unit 13 refers to the character score storage unit33 to specify how many scores of the degree of reliability are added tothe scene. The scene discrimination unit 13 adds the specified pointscore to the degree of reliability outputted in each scene. A specificexample will be described below.

FIG. 9 is a view showing a specific example a correspondence tableexpressing an association between the keyword, scene, and the pointscore, which are stored in the character score storage unit 33. As shownin FIG. 9, the scene that becomes the point target and the point scoreare stored for every keyword in the correspondence table whilecorrelated with each other.

For example, in a first record of the image data shown in FIG. 9, thecase where the keyword “

” is included in the character region of the image data means that thescore “50” is added to the degree of reliability of the scene“landscape” in the discrimination result (see FIG. 8) of the image data.

More particularly, the keyword extracting unit 17 extracts the total ofseven keywords shown in FIG. 7, such as one keyword “

” and two keywords “

”, which are recognized by the character recognition unit 14, from thecharacter strings of the three character regions. As shown in FIG. 7,the keyword extracted by the keyword extracting unit 17 may be storedwhile correlated with the character region or the keyword maycollectively be stored while correlated with the image data(Fujiyama.jpg).

After outputting the scene discrimination result of FIG. 8 through thescene discrimination processing described in the first embodiment, thescene discrimination unit 13 refers to the correspondence table (FIG. 9)of the character score storage unit 33 based on the keyword extracted bythe keyword extracting unit 17.

The scene discrimination unit 13 adds the point score “50 points×1=50points” of the keyword “

” to the degree of reliability of the scene “landscape”. The scenediscrimination unit 13 adds the point score “10 points×2=20 points” ofthe keyword “

” to the degree of reliability of the scene “landscape”. The scenediscrimination unit 13 does not add the point score for the fourkeywords except the keywords “

” and “

” when the four keywords are not stored in the character score storageunit 33. That is, the scene discrimination unit 13 adds the point score“70” to the score “60” of the degree of reliability of the scene“landscape” of FIG. 8 to obtain the final score “130”, and the scenediscrimination unit 13 outputs the final score “130” of the degree ofreliability of the scene “landscape”.

Thus, the scene discrimination result can be outputted with higherreliability by adding the semantic content of the character stringincluded in the image data to the scene discrimination result of theimage data. Frequently, the word associated deeply with the scene of theimage data is included in the character string such as the telop and thephotograph title, and therefore the accuracy of the scene discriminationresult can be improved by setting the word to one of the indexes of thescene discrimination processing.

[Scene Discrimination Processing Flow]

FIGS. 10A and 10B are flowcharts each showing an example of the scenediscrimination processing flow of the digital photo frame 100 of thesecond embodiment.

Similarly to the method in S101 to S103 of FIG. 6, the image attributediscrimination apparatus 10 reads the image data (Fujiyama.jpg) thatbecomes the processing target from the image data storage unit 30 toexpand the image data (Fujiyama.jpg) in the image memory 5 a (S201). Thetarget pixel specifying unit 12 defines the flag array of each pixel ofthe expanded image data to initialize the flag array to TRUE (S202). Thecharacter region extracting unit 11 extracts the character region as theheterogeneous region with respect to the image data expanded in theimage memory 5 a (S203). As shown in FIG. 3B, the character regionextracting unit 11 extracts the three character regions of the firstcharacter region Ar1 to the third character region Ar3.

The target pixel specifying unit 12 specifies whether each pixelbelonging to the extracted character region becomes the pixel of thefeature quantity extracting target for the scene discrimination.Particularly, the target pixel specifying unit 12 determines whetherareas of all the character regions extracted from the image data by thecharacter region extracting unit 11 is not lower than a predeterminedthreshold or lower than a predetermined threshold (S204). When the totalarea of all the character regions is lower than the predeterminedthreshold (NO in S204), it is considered that the character region(heterogeneous region) has a small adverse effect on the scenediscrimination processing of the image data, the target pixel specifyingprocessing is not performed, all the pixels of the image data are set tothe analysis target, and the flow goes to the scene discriminationprocessing (FIG. 10B) of S211 and thereafter.

On the other hand, when the total area of all the character regions isnot lower than the predetermined threshold (YES in S204), the flow goesto the target pixel specifying processing from S205 and thereafter. InS205, the target pixel specifying unit 12 substitutes an initial value 1for a variable i. The processing of maintaining the flag TRUE or theprocessing of changing the flag to FALSE is performed to each pixel ofthe ith character region.

Specifically, the character recognition unit 14 performs the characterrecognition processing to the ith character region (S206). As shown inFIG. 7, the character recognition unit 14 outputs the characterrecognition result of the ith character region and the degree ofreliability of the character recognition result. In this case, thekeyword extracting unit 17 may extract the keyword from the character(string) recognized by the character recognition unit 14. Alternatively,the keywords may collectively be extracted at the end after thecharacter recognition processing is completed for all the regions.

The target pixel specifying unit 12 refers to the degree of reliabilityof the character recognition result in the character region, outputtedfrom the character recognition unit 14, to determine whether the degreeof reliability of the character recognition result is not lower than thepredetermined threshold or lower than a predetermined threshold (S207).When the degree of reliability of the character recognition result islower than the predetermined threshold (NO in S207), the target pixelspecifying unit 12 determines that there is a high possibility that theith region regarded as the character region is actually not thecharacter region (that is, the ith region does not include theheterogeneous matter), and the target pixel specifying unit 12 leavesthe flag of each pixel in the region TRUE. That is, the target pixelspecifying unit 12 determines that each pixel of the region is notexcluded from the analysis target for the scene discrimination.

On the other hand, when the degree of reliability of the characterrecognition result is not lower than the predetermined threshold (YES inS207), the target pixel specifying unit 12 determines that there is ahigh possibility that the character region includes the character(string) that has the adverse effect on the scene discrimination, andthe target pixel specifying unit 12 sets the flag of each pixel in thecharacter region to FALSE (S208). That is, the target pixel specifyingunit 12 determines that each pixel of the character region is excludedfrom the analysis target for the scene discrimination. When the flag ofeach pixel is specified as TRUE or FALSE with respect to one characterregion, the target pixel specifying unit 12 increments i by one (S209),the target pixel specifying unit 12 specifies whether the next characterregion becomes the analysis target pixel in the similar procedure, andthe target pixel specifying unit 12 repeats the processing for all thecharacter regions extracted by the character region extracting unit 11.When the target pixel specifying unit 12 ends the target pixelspecifying processing for all the character regions (for example, allthe three character regions) (S210), the scene discrimination unit 13performs the scene discrimination processing to the image data(Fujiyama.jpg).

The scene discrimination unit 13 extracts the feature quantity (producesthe histogram) only from the pixel whose flag satisfies the condition“TRUE” in the image data by the method similar to that in S105 and S106of FIG. 6 (S211), and checks the extracted feature quantity against themodel feature quantity (model histogram) of each scene to discriminatethe scene of the image data (S212). The scene discrimination unit 13computes the degree of reliability for the scene discriminated as thescene of the image data and the scenes from the second-place scene,based on factors such as the degree of similarity between the featurequantity of the image data and the model feature quantity of each scene,the keyword included in the character region obtained by the characterrecognition unit 14 and keyword extracting unit 17, and the size of thecharacter region out of the target (S213). For example, the scenediscrimination unit 13 outputs the scene discrimination result and thedegree of reliability of the scene discrimination result like “(firstplace) scene: landscape, degree of reliability: 130”.

The scene discrimination unit 13 determines how much likelihood that thescene of the image data (Fujiyama.jpg) is the discriminated scene (forexample, “landscape”). For example, the scene discrimination unit 13determines whether the “degree of reliability: 130” is not lower than apredetermined threshold or lower than a predetermined threshold (S214).When the degree of reliability of the scene discrimination result is notlower than predetermined threshold, the scene discrimination unit 13determines that the discriminated scene is almost certainly correct, andoutputs the scene discrimination result to the scene-by-scene processingperforming unit 16. For example, the scene discrimination unit 13outputs the discrimination result that the scene of the image data(Fujiyama.jpg) is the “landscape” to the scene-by-scene processingperforming unit 16 (S218).

On the other hand, when the degree of reliability of the scenediscrimination result is lower than predetermined threshold, the scenediscrimination unit 13 determines that it is doubtful that the scene ofthe image data is actually the scene, and the image attributediscrimination apparatus 10 performs the processing of improving theaccuracy of the discrimination result. Specifically, the restorationunit 15 performs the restoration processing of removing theheterogeneous matter to each pixel in which the target pixel specifyingunit 12 sets the flag to FALSE (S215). A well-known restorationtechnique is applied to this restoration processing.

The scene discrimination unit 13 extracts the feature quantity from eachpixel having the flag “FALSE”, which is restored by the restoration unit15 (S216). The scene discrimination unit 13 combines the histogram ofeach pixel having the flag “FALSE” and the histogram of each pixelhaving the flag “TRUE”, which is produced in S211, and performs thematching between the combined histogram and the model histogram of eachscene to perform the scene discrimination of the image data again(S217). Therefore, the scene discrimination result of the image data(Fujiyama.jpg) and the degree of reliability of the scene discriminationresult are determined again, and the discrimination unit 13 outputs thescene having the highest degree of reliability as the scene of the imagedata to the scene-by-scene processing performing unit 16 (S218).

Therefore, the scene-by-scene processing performing unit 16 can performthe processing to the image data (Fujiyama.jpg) according to the scene“landscape”. For example, the scene-by-scene processing performing unit16 has the scene-by-scene automatic correction function, and thescene-by-scene processing performing unit 16 can perform the imageprocessing to the image data such that the landscape photograph isdisplayed the most beautiful, and display the image data on the displayunit 3.

Thus, according to the above method, a balance between the efficiency ofthe scene discrimination processing and the improvement of the accuracyof the scene discrimination processing can be established in the imageattribute discrimination apparatus 10 according to the performance andusage environment of the image attribute discrimination apparatus 10.

The image attribute discrimination apparatus 10 of one or moreembodiments of the present invention need not include all the additionalconfigurations of the second embodiment. In consideration of theinformation processing ability, usage, and usage environment of theapparatus that realizes one or more embodiments of the presentinvention, the configuration of the image attribute discriminationapparatus 10 is selectively designed as appropriate such that theefficiency of the scene discrimination processing and the improvement ofthe accuracy of the scene discrimination processing can be realized in abalanced manner.

[Method for Extracting Character Region]

In the above embodiments, as shown in FIG. 3B, the character regionextracting unit 11 extracts the character-string-like circumscribedrectangle that collects to some extent as the character region from theimage data. However, the character region extracting unit 11 of one ormore embodiments of the present invention is not limited to the aboveembodiments.

FIGS. 11 and 12 are views showing another example of the region that isextracted by the character region extracting unit 11 as the characterregion from the image data of FIG. 3A.

For example, as shown in FIG. 11, the character region extracting unit11 may extract the character-like circumscribed rectangle as thecharacter region in one character unit. In such cases, disadvantageouslythe number of regions is increased to apply the processing load on theimage attribute discrimination apparatus 10 when the target pixelspecifying unit 12 performs the target pixel specifying processing. Atthe same time, the number of pixels that are ignored as the pixel out ofthe analysis target can be decreased rather than the case where thecharacter region is largely classified into the three regions as shownin FIG. 3B. Accordingly, advantageously the accuracy of the scenediscrimination processing can be improved.

Alternatively, as shown in FIG. 12, the character region extracting unit11 may exactly extract not the circumscribed rectangle of theheterogeneous matter but only the pixels in which the heterogeneousmatter (such as the character) is photographed from the color ortexture. In such cases, while the processing load is further increasedin the character region extracting unit 11 and target pixel specifyingunit 12, the number of pixels that are ignored as the pixel out of theanalysis target can largely be decreased, and therefore the accuracy ofthe scene discrimination processing can further be improved.

Third Embodiment

In the first and second embodiments, the image attribute discriminationapparatus 10 can accurately and efficiently discriminate the attribute(scene) of the image data irrespective of the heterogeneous region ofthe image data. The image attribute discrimination apparatus 10discriminates the scene by checking the model feature quantity that ispreviously learned by the scene feature quantity storage unit 31 againstthe feature quantity of the image data. Accordingly, in order to performthe accurate scene discrimination, it is necessary that the modelfeature quantity correctly reflect the feature according to the scene.An attribute discrimination support apparatus 20 according to a thirdembodiment of the present invention more precisely produces the modelfeature quantity of each scene, which is learned by the scene featurequantity storage unit 31.

The attribute discrimination support apparatus 20 of one or moreembodiments of the present invention performs a scene learning function.In the scene learning function, the attribute discrimination supportapparatus 20 receives image data of a sample that becomes a learningtarget while correlating the image data with a correct scene, extractsthe feature quantity from the image data, and learns the featurequantity as part of the model feature quantity of the designated scene.For example, a plurality of pieces of image data whose scenes arecategorized into the “landscape” are previously prepared, the featurequantities are extracted from the pieces of image data, and an averagevalue of the feature quantities is used as the model feature quantity ofthe scene “landscape”.

Accordingly, when the image data inputted as the sample includes theheterogeneous region (the character string such as the telop), the modelfeature quantity includes the feature different from the originalfeature of the scene. For example, usually the image data “landscape”does not include the yellow character shape, but the heterogeneousyellow text object deforms the original model feature quantity in anincorrect direction. Unless the model feature quantity correctlyreflects the feature according to the scene, unfortunately the accuracyof the scene discrimination processing is degraded when the processingis performed using the model feature quantity.

Therefore, when the heterogeneous region is included in the image dataof the inputted sample, the attribute discrimination support apparatus20 of one or more embodiments of the present invention determines thefeature quantity after detecting and removing the heterogeneous region,and the attribute discrimination support apparatus 20 adds the featurequantity to the model feature quantity of the designated scene.Therefore, the precise scene feature quantity can be producedirrespective of the heterogeneous region of the image data, and theimage attribute discrimination apparatus 10 can accurately discriminatethe attribute of the image data irrespective of the heterogeneousregion.

The attribute discrimination support apparatus 20 may be applied tovarious pieces of image processing apparatus such as the digital photoframe 100 used by the user. Alternatively, the attribute discriminationsupport apparatus 20 of one or more embodiments of the present inventionmay be implemented by an information processing apparatus that producesthe model feature quantity stored in the scene feature quantity storageunit 31 of the image processing apparatus based on a large amount ofsample image data in a production stage of the image processingapparatus.

[Configuration of Digital Photo Frame]

FIG. 13 is a block diagram showing a configuration of a main part of adigital photo frame 100 of the third embodiment of the presentinvention. The reference numeral of each constituent of FIG. 13corresponds to the reference numeral of each constituent of FIG. 1, andthe same reference numeral expresses the same constituent. Accordingly,the overlapping description of the constituent that is already describedin the first and second embodiments will not be given.

The digital photo frame 100 of the third embodiment differs from thedigital photo frame 100 shown in FIG. 1 in that the control unit 1further includes a model feature quantity computing unit 18 as afunctional block. The scene feature quantity computing unit 18 and otherfunctional blocks (particularly, the character region extracting unit11, the target pixel specifying unit 12, and the character recognitionunit 14) act as the attribute discrimination support apparatus 20 thatperforms the scene learning function. The attribute discriminationsupport apparatus 20 may further include the restoration unit 15.

The attribute discrimination support apparatus 20 receives the imagedata of the sample that becomes the learning target while correlatingthe image data with the designated correct scene. There is no particularlimitation to the method for receiving the input. For example, the userloads an external recording medium in which the image data to be learnedis recorded on the digital photo frame 100, and the digital photo frame100 captures the image data through an external interface (not shown).The user manipulates the digital photo frame 100 using the manipulationunit 4, designates the correct scene correlated with the captured imagedata, and issues an instruction to perform the learning. The attributediscrimination support apparatus 20 registers the received image data tothe image data storage unit 30 while correlating the image data with theinputted correct scene. The registered image data may be used as theimage data displayed on the display unit 3 while used in the scenelearning processing.

When the learning instruction is issued, the character region extractingunit 11 processes the image data that is received as the learningtarget, and extracts the heterogeneous region (in this case, thecharacter region) when the heterogeneous region is included in the imagedata.

The target pixel specifying unit 12 specifies whether each pixel in thecharacter region extracted by the character region extracting unit 11becomes the target pixel of the feature quantity extraction. Similarlyto the above embodiments, the target pixel specifying unit 12 sets theflag of the target pixel to TRUE and sets the flag of the pixel out ofthe analysis target to FALSE.

The model feature quantity computing unit 18 extracts the featurequantity of the image data that is received as the learning target, andcomputes the model feature quantity of the designated scene using theextracted feature quantity. In the digital photo frame 100 of the thirdembodiment, the model feature quantity computing unit 18 re-computes theaverage value of the feature quantity while the newly-extracted featurequantity is included in the already-produced model feature quantity, andthe model feature quantity computing unit 18 updates the model featurequantity of the designated scene.

For example, assuming that X is a model feature quantity of the scene“landscape” at the present moment, N is the number of (featurequantities) of the pieces of sample image data of the “landscape” thatis the origin of the model feature quantity X, and Y is a featurequantity of the newly-extracted image data A, when image data A of thelearning target is inputted while the scene “landscape” is designated,the model feature quantity computing unit 18 produces the model featurequantity of the new “landscape” from the following equation, and themodel feature quantity computing unit 18 updates the model featurequantity of the scene feature quantity storage unit 31:(X*N+Y)/(N+1)  (equation 1)The equation 1 is applied to the case where the number of pieces of dataof the feature quantity Y is 1, that is, one piece of image data A. Thefeature quantity X and the feature quantity Y are vector quantities. Forexample, the feature quantity X and the feature quantity Y indicate thehistograms.

According to the configuration, in performing the scene learningfunction, the character region extracting unit 11 performs the characterregion extracting processing as pre-processing to the image data thatbecomes the learning target. The model feature quantity computing unit18 produces the model feature quantity based on the feature quantitythat is obtained by excluding the pixel out of the processing targetspecified by the target pixel specifying unit 12.

Therefore, the model feature quantity in which the adverse effect of theheterogeneous matter is removed can be obtained even if the image dataincluding the heterogeneous matter (the character such as the telop) isinconveniently mixed in the image data of the learning target. As aresult, the scene discrimination accuracy of the image attributediscrimination apparatus 10 can be improved.

The restoration unit 15 may perform the restoration processing to thepixel that is excluded from the feature quantity extracting target bythe target pixel specifying unit 12. For example, the restoration unit15 can restore the original background hidden behind the characterstring. Therefore, higher-reliability model feature quantity can beproduced.

[Scene Learning Processing Flow]

FIG. 14 is a flowchart showing a scene learning processing flow of thedigital photo frame 100 of the third embodiment.

The attribute discrimination support apparatus 20 receives thedesignation of the correct scene (set to “landscape”, in this case)correlated with the image data along with the input of the image data(set to Fujiyama.jpg shown in FIG. 3A, also in this case) that becomesthe learning target (S301).

The character region extracting unit 11 performs the heterogeneousregion (character region, in this case) extracting processing to theimage data (Fujiyama.jpg) (S302). The character region extractingprocessing is performed in the procedure similar to that of the firstand second embodiments. For example, as shown in FIG. 3B, it is assumedthat the three character regions of the first character region Ar1 tothird character region Ar3 are extracted.

The target pixel specifying unit 12 performs the target pixel specifyingprocessing to each pixel belonging to the extracted character region inorder to specify whether the pixel belonging to the extracted characterregion becomes the pixel of the feature quantity extracting target forthe scene learning (S303). In the third embodiment, whether each pixelof the character region becomes the feature quantity extracting target(TRUE) or does not become the feature quantity extracting target (FALSE)is specified in the procedure similar to that in S205 to S210 of FIG.10A. That is, the target pixel specifying unit 12 sets each pixel in thecharacter region to the pixel of the feature quantity extracting targetwhen there is a high possibility that the character region does notactually include the character, and the target pixel specifying unit 12sets each pixel in the pixel out of the feature quantity extractingtarget when there is a high possibility that the character regionincludes the character.

The model feature quantity computing unit 18 extracts feature quantity(for example, produces the histogram) only from the pixel whose flagsatisfies the condition “TRUE” in the pixels of the image data(Fujiyama.jpg) (S304). The model feature quantity computing unit 18reads the model feature quantity of the scene (“landscape”, in thiscase) received in S301 from the scene feature quantity storage unit 31,and the model feature quantity computing unit 18 re-computes the modelfeature quantity based on the feature quantity extracted in S304 toupdate the model feature quantity (S305). For example, the model featurequantity computing unit 18 computes the average value of the colorhistograms obtained from the pieces of sample image data of thelandscape including the image data (Fujiyama.jpg), and updates the colorhistogram as the new model histogram of the scene “landscape”.

According to the above method, in performing the scene learningfunction, when the image data that becomes the learning target includesthe heterogeneous region such as the character (string), the featurequantity obtained from the pixel group except the heterogeneous regioncan be added to the model feature quantity of the designated scene.

Therefore, the model feature quantity can be produced more precisely, sothat the image attribute discrimination apparatus 10 can accuratelyperform the scene discrimination.

In the second embodiment, when the extracted character region is small,the target pixel specifying processing (processing of determiningwhether the flag is set to TRUE or FALSE) is omitted in order to performefficiently the scene discrimination processing. However, in the thirdembodiment, even if the small character region is extracted from theimage data, when the extracted character region is the heterogeneousregion, preferably the flag is set to FALSE to exclude the heterogeneousregion from the target pixel. This is attributed to the fact that, whilethe small character region has the small adverse effect when the correctscene is discriminated for one piece of image data in the secondembodiment, accumulation of the small character regions possiblyobstructs the production of the precise model feature quantity when themodel feature quantity of one scene is produced using many pieces ofimage data in the third embodiment.

The attribute discrimination support apparatus 20 may further includethe restoration unit 15. The restoration unit 15 performs therestoration processing to the character region, when the area (area ofcharacter region) of the pixels having the flag “FALSE” specified bytarget pixel specifying unit 12 in S303 is larger than a predeterminedthreshold. The target pixel specifying unit 12 sets the flag of thepixels in the restored region to TRUE to enlarge the area of the targetpixel. Therefore, more identical pixels can be set to the featurequantity extracting target, and the reliability of the produced modelfeature quantity can further be enhanced.

FIG. 15 is a block diagram showing a configuration of a main part of anattribute discrimination support apparatus 20 that produces the modelfeature quantity mounted on the scene feature quantity storage unit 31of the digital photo frame 100 of the third embodiment. The attributediscrimination support apparatus 20 is realized by various pieces ofinformation processing apparatus such as a server suitable to process alarge amount of image data, a personal computer, and a super computer.The reference numeral of each constituent of FIG. 15 corresponds to thereference numeral of each constituent of FIGS. 1 and 13, and the samereference numeral expresses the constituent having the same function.Accordingly, the overlapping description of the constituent that isalready described in the embodiments will not be given.

The display unit 3 displays the manipulation screen as the GUI(Graphical User Interface) screen in order that the user registers alarge amount of image data or designates the scene. For example, a listof icons is displayed in order to manipulate the image data of thelearning target, the image data registered in the image data storageunit 30 is displayed in the thumbnail form, or the GUI screen isdisplayed such that the user conveniently performs the scene learningfunction.

The manipulation unit 4 is used when the user manipulates the attributediscrimination support apparatus 20. For example, the manipulation unit4 is realized by a mouse and a keyboard. Specifically, the usermanipulates the mouse to collectively select the pieces of image data ofmany newly-registered samples displayed on the display unit 3, and theuser can store the pieces of image data in a folder of the specificscene “landscape” by drag and drop. Therefore, the user can designatethe scene to register the large amount of image data at one time, andthe user can cause the attribute discrimination support apparatus 20 tolearn the feature of the scene by a simple manipulation.

The control unit 1 performs various arithmetic operations by performingthe program that is read in the temporary storage unit 5 from thestorage unit 2, and the control unit 1 wholly controls the units of theattribute discrimination support apparatus 20 through the bus 6. Thecontrol unit 1 includes at least a heterogeneous region extracting unit11 a, the target pixel specifying unit 12, and the model featurequantity computing unit 18 as the functional blocks. The control unit 1may further include a learning target management unit 19, the characterrecognition unit 14, and the restoration unit 15. Each of the functionalblocks performs the scene learning function of the attributediscrimination support apparatus 20. Each functional block of thecontrol unit 1 can be realized, such that the CPU (Central ProcessingUnit) reads a program stored in the storage unit 2 implemented by theROM (Read Only Memory) to the temporary storage unit 5 implemented bythe RAM (Random Access Memory) and executes the program.

The heterogeneous region extracting unit 11 a extracts the heterogeneousregion included in the image data of the learning target. In the aboveembodiments, the character region extracting unit 11 extracts thecharacter region including the character (string) as the heterogeneousregion. However, the heterogeneous region extracting unit 11 a isconfigured to extract not only the character region but also theheterogeneous region including any heterogeneous matter. A specificexample of the heterogeneous region except the character region isdescribed later. In the attribute discrimination support apparatus 20 ofthe third embodiment, not only the character (string) but also the imagedata including any heterogeneous matter that is not suitable to thesample are possibly mixed when the large amount of sample image data isread at one time in order to produce the model feature quantity.Therefore, preferably the heterogeneous region extracting unit 11 a candetect any kind of the heterogeneous matter from the features such asthe color and texture.

The learning target management unit 19 receives the learning instructionfrom the user. The learning target management unit 19 manages the largeamount of image data of the learning target inputted thereto along withthe information on the designated scene while storing the image data ofthe learning target and the information on the designated scene in theimage data storage unit 30. In extracting the feature quantity, theimage data stored by the learning target management unit 19 is expandedone by one on the image memory 5 a by the model feature quantitycomputing unit 18. The learning target management unit 19 transmits whatis correct scene of the expanded image data to the model featurequantity computing unit 18.

The model feature quantity computing unit 18 extracts the featurequantities of the plurality of pieces of sample image data from onescene inputted thereto in the similar procedure, and computes the modelfeature quantity based on the feature quantities.

For example, when the pieces of image data of 100 samples are inputtedalong with the learning instruction while correlated with the scene“landscape”, the learning target management unit 19 stores the 100pieces of image data in the image data storage unit 30 while correlatingthe 100 pieces of image data with the scene “landscape”. Theheterogeneous region extracting unit 11 a detects the heterogeneousregion of one piece of image data expanded on the image memory 5 a, andthe target pixel specifying unit 12 sets the flag (FALSE) to each pixelin order to exclude the heterogeneous region.

The model feature quantity computing unit 18 extracts the featurequantity only from the pixel having the flag “TRUE” with respect to theimage data. The model feature quantity computing unit 18 produces theaverage value of the feature quantities of all the 100 pieces of imagedata obtained in the similar manner as the model feature quantity of thescene “landscape”. The model feature quantity produced by the modelfeature quantity computing unit 18 is once stored in the scene featurequantity storage unit 31, and the model feature quantity is mounted oneach digital photo frame 100 in production process by appropriate means.

According to the configuration, even if the image data including theheterogeneous region is included in the sample used to produce the modelfeature quantity, the image data including the heterogeneous region isexcluded, and the model feature quantity is produced based on thefeature obtained from the identical pixel. The precise model featurequantity suitable to the designated scene can be mounted on the digitalphoto frame 100, and therefore the digital photo frame 100 canaccurately perform the scene discrimination processing.

As described above, the heterogeneous region extracting unit 11 adetects not only the character (string) but also various heterogeneousmatters, and the heterogeneous region extracting unit 11 a can extractthe heterogeneous region including the heterogeneous matters. FIGS. 16to 19 show specific examples of various heterogeneous regions.

FIG. 16 is a view showing an example of the image data in which a smear(a white-spot region in a broken-line frame) is generated due to anenvironment in taking the photograph or subject state. FIG. 17 is a viewshowing an example of the image data in which the objects such asgraffiti (hand writing edit with a touch pen), an illustration, and astamp are merged in the image edit process. FIG. 18 is a view showing anexample of the image data in which an original is scanned while an endof the original is folded. FIG. 19 is a view showing an example of theimage data in which a finger of a photographer is taken in a photographupon taking the photograph.

The heterogeneous region extracting unit 11 a detects that the attributeof the region in the broken-line frame is different from that of otherregions based on the difference in color or texture, and theheterogeneous region extracting unit 11 a extracts the detected regionas the heterogeneous region.

According to the configuration, when extracting the feature quantity,the model feature quantity computing unit 18 can deal with variousheterogeneous matters that have the adverse effect on the scenediscrimination, and the model feature quantity is computed while variousheterogeneous matters are removed. Accordingly, the model featurequantity can be obtained more precisely, and therefore the accuracy ofthe scene discrimination result of the image attribute discriminationapparatus 10 can further be improved.

The image attribute discrimination apparatus 10 may include theheterogeneous region extracting unit 11 a. In such cases, even if theheterogeneous matter is included in the image data in addition to thecharacter, the scene discrimination processing can correctly beperformed irrespective of the heterogeneous matter.

One or more embodiments of the present invention are not limited to theabove embodiments, but various changes can be made without departingfrom the scope of the invention. An embodiment obtained by appropriatelycombining technical means disclosed in different embodiments are alsoincluded in the technical range of the invention.

Finally, each block of the image attribute discrimination apparatus 10and attribute discrimination support apparatus 20, particularly theheterogeneous region extracting unit 11 a, the character regionextracting unit 11, the target image specifying unit 12, the scenediscrimination unit 13, and the model feature quantity computing unit 18may be formed by hardware logic or may be realized as follows bysoftware using the CPU as follows.

That is, each of the image attribute discrimination apparatus 10 and theattribute discrimination support apparatus 20 includes the CPU (CentralProcessing Unit) that executes a command of a control program realizingeach function, the ROM (Read Only Memory) in which the program isstored, the RAM (Random Access Memory) in which the program is expanded,and the storage device (recording medium) such as a memory in which theprogram and various pieces of data are stored. Program codes (anexecutable format program, an intermediate code program, and a sourceprogram) of the control programs that are the software realizing thefunctions in the image attribute discrimination apparatus 10 (or theattribute discrimination support apparatus 20) are recorded in therecording medium while the computer can read the program codes, therecording medium is supplied to the image attribute discriminationapparatus 10 (or the attribute discrimination support apparatus 20), andthe computer (or the CPU or MPU) reads and executes the program coderecorded in the recording medium.

Examples of the recording medium include tape system such as magnetictape and cassette tape, disk systems including magnetic disks such asfloppy disk (registered trademark) and a hard disk and optical diskssuch as a CD-ROM, an MO, an MD, a DVD, and a CD-R, card systems such asan IC card (including a memory card) and an optical card, andsemiconductor memory systems such as a mask ROM, an EPROM, an EEPROM anda flash ROM.

The image attribute discrimination apparatus 10 (or the attributediscrimination support apparatus 20) is configured to be able to beconnected to a communication network, and the program code may besupplied through the communication network. There is no particularlimitation to the transmission medium constituting the communicationnetwork. Examples of the communication network include the Internet, anintranet, an extranet, a LAN, an ISDN, a VAN, a CATV communicationnetwork, a virtual private network, a telephone line network, a mobilecommunication network, and a satellite communication network. Examplesof the transmission medium include a wired medium such as IEEE1394, USB,power-line carrier, a cable TV line, a telephone line, and an ADSL lineand a wireless medium such as an infrared ray such as IrDA and a remotecontroller, Bluetooth (registered trademark), 802.11 wireless, HDR, amobile telephone network, a satellite line, and a terrestrial digitalnetwork. One or more embodiments of the present invention can also berealized by a mode of a computer data signal buried in a carrier wave,in which the program code is implemented by electronic transmission.

According to the image attribute discrimination apparatus and attributediscrimination support apparatus of one or more embodiments of thepresent invention, the attribute of the image data is accuratelydiscriminated, so that the image attribute discrimination apparatus andthe attribute discrimination support apparatus can suitably be appliedto various pieces of image processing apparatus that perform theprocessing to the image data attribute by attribute according to thediscrimination result of the attribute. For example, one or moreembodiments of the present invention can be used in the digital photoframe, the digital camera, the digital video camera, the digital videorecorder/player, the digital television, the personal computer, theprinter, and the scanner.

What is claimed is:
 1. An image attribute discrimination apparatus thatdiscriminates an attribute of image data based on a content produced bythe image data, the image attribute discrimination apparatus comprising:a heterogeneous region specifying hardware logic that specifies acharacter region including a character as a heterogeneous region fromthe image data of a still image based on a determination whether theimage data of the still image contains the heterogeneous region, whereinthe heterogeneous region comprises a heterogeneous matter whoseattribute is different from that of the content originally produced bythe image data; an attribute discrimination hardware logic that, if theimage data contains the heterogeneous region, discriminates theattribute of the image data based on a feature quantity extracted from apixel group except each pixel in the heterogeneous region in each pixelof the image data and excludes data of the heterogeneous region from theremaining of the image data, wherein the attribute is a characteristicof a scene shown by the image data; a restoration hardware logic thatrestores a pixel identical to a target pixel with respect to anout-of-target pixel region in each pixel of the image data, wherein theout-of-target region does not become a feature quantity extractingtarget; and a target pixel determination hardware logic that determineswhether each pixel in the heterogeneous region specified by theheterogeneous region specifying hardware logic is the target pixel whosefeature quantity is extracted by the attribute discrimination hardwarelogic, wherein the attribute discrimination hardware logic discriminatesthe attribute of the image data based on the feature quantity extractedfrom the pixel group except the out-of-target pixel in each pixel of theimage data, and the out-of-target pixel is determined to be out of thetarget by the target pixel determination hardware logic.
 2. The imageattribute discrimination apparatus according to claim 1, wherein theheterogeneous region specifying hardware logic specifies the characterregion including the character as the heterogeneous region, wherein theimage attribute discrimination apparatus further includes a characterrecognition hardware logic that recognizes the character in thecharacter region specified by the heterogeneous region specifyinghardware logic, and wherein the target pixel determination hardwarelogic that determines the pixel in the character region as the targetpixel, when a degree of reliability of a character recognition result isnot more than a predetermined value, wherein the degree of reliabilityindicates likelihood that the character in the character region is thecharacter recognized by the character recognition hardware logic.
 3. Theimage attribute discrimination apparatus according to claim 2, furthercomprising: a keyword extracting hardware logic that extracts a keyword,the character or character string recognized by the characterrecognition hardware logic; and a word association storage hardwarelogic in which association between each keyword extracted by the keywordextracting hardware logic and each attribute discriminated by theattribute discrimination hardware logic is stored, wherein the attributediscrimination hardware logic refers to the word association storagehardware logic, and wherein the attribute discrimination hardware logicdiscriminates the attribute of the image data in consideration of alevel of association between the keyword extracted from the characterregion of the image data and each attribute.
 4. The image attributediscrimination apparatus according to claim 3, wherein the attributediscrimination hardware logic checks the feature quantity of the imagedata against a model feature quantity that is previously defined in eachplurality of kinds of attributes, wherein the attribute discriminationhardware logic discriminates the attribute of the image data bycomputing a degree of reliability of an attribute discrimination resultaccording to a degree of similarity between the feature quantity of theimage data and the model feature quantity, the degree of reliabilityindicating likelihood that the attribute of the image data is theattribute, and wherein the association between the keyword and theattribute is stored in the word association storage hardware logic as ascore added to the degree of reliability of the attribute discriminationresult.
 5. The image attribute discrimination apparatus according toclaim 4, wherein the restoration hardware logic performs the restorationwhen the degree of reliability of the attribute discrimination result islower than a predetermined value.
 6. The image attribute discriminationapparatus according to claim 4, wherein the attribute discriminationhardware logic computes the degree of reliability lower with increasingregion of the out-of-target pixel that does not becomes the featurequantity extracting target in each pixel of the image data.
 7. The imageattribute discrimination apparatus according to claim 1, wherein thetarget pixel determination hardware logic determines each pixel in theheterogeneous region as the out-of-target pixel only when an areaoccupied by the heterogeneous region in the image data is more than apredetermined value.
 8. The image attribute discrimination apparatusaccording to claim 1, further comprising: a model feature quantitycomputing hardware logic that computes a model feature quantity of adesignated attribute using the feature quantity extracted from the pixelgroup except each pixel in the heterogeneous region specified by theheterogeneous region specifying hardware logic in each pixel of theimage data, when image data and the designation of the attribute of theimage data are inputted to the image attribute discrimination apparatus,wherein the attribute discrimination hardware logic checks the featurequantity of the image data against the model feature quantity computedin each attribute by the model feature quantity computing hardwarelogic, and wherein the attribute discrimination hardware logicdiscriminates the attribute of the image data according to a degree ofsimilarity between the feature quantity of the image data and the modelfeature quantity.
 9. An image attribute discrimination method fordiscriminating an attribute of image data based on a content produced bythe image data, the image attribute discrimination method comprising thesteps of: specifying a character region including a character as aheterogeneous region from the image data of a still image based on adetermination whether the image data of the still image contains theheterogeneous region, wherein the heterogeneous region further comprisesa heterogeneous matter whose attribute is different from that of thecontent originally produced by the image; discriminating the attributeof the image data based on a feature quantity extracted from a pixelgroup except each pixel in the heterogeneous region in each pixel of theimage data and excluding data of the heterogeneous region from theremaining of the image data, if the image data contains theheterogeneous region; and restoring a pixel identical to a target pixelwith respect to an out-of-target pixel region in each pixel of the imagedata, wherein the out-of-target region does not become a featurequantity extracting target, wherein the image data is stored in a datastorage hardware logic and the content produced by the image data isdisplayed by a display device, wherein the attribute is a characteristicof a scene shown by the image data, and wherein the image attributediscrimination method further comprises determining whether each pixelin the specified heterogeneous region is the target pixel whose featurequantity is extracted in the discriminating, wherein the discriminatingdiscriminates the attribute of the image data based on the featurequantity extracted from the pixel group except the out-of-target pixelin each pixel of the image data, the out-of-target pixel is determinedto be out of the target in the determining.
 10. A non-transitorycomputer readable medium comprising a control program that causes acomputer to perform steps of: specifying a character region including acharacter as a heterogeneous region from the image data of a still imagebased on a determination whether the image data of the still imagecontains the heterogeneous region, wherein the heterogeneous regionfurther comprises a heterogeneous matter whose attribute is differentfrom that of the content originally produced by the image;discriminating the attribute of the image data based on a featurequantity extracted from a pixel group except each pixel in theheterogeneous region in each pixel of the image data and excluding dataof the heterogeneous region from the remaining of the image data, if theimage data contains the heterogeneous region; and restoring a pixelidentical to a target pixel with respect to an out-of-target pixelregion in each pixel of the image data, wherein the out-of-target re iondoes not become a feature quantity extracting target, wherein the imagedata is stored in a data storage hardware logic and the content producedby the image data is displayed by a display device, wherein theattribute is a characteristic of a scene shown by the image data, andwherein the control program causes the computer to further performdetermining whether each pixel in the specified heterogeneous region isthe target pixel whose feature quantity is extracted in thediscriminating, wherein the discriminating discriminates the attributeof the image data based on the feature quantity extracted from the pixelgroup except the out-of-target pixel in each pixel of the image data,and the out-of-target pixel is determined to be out of the target in thedetermining.